International Journal of Information and Computer Science 

IJICS Volume 1 , Issue 2, May 201 2 PP. 39-45 
ISSN(online) 2161-5381 
ISSN(print) 2161-6450 
www.iji-cs.org 



Acclimatization Calculation Research Based on 

Multiple Stereovision 

Yadan Pei 1 Qiuqi Ruan 2 
Institute of Information Science ,Beijing Jiaotong University ,Beijing,China 
Email: 10120395@bjtu.edu.cn 

(Abstract) As environment factors such as illumination, posture greatly affect the identification results on face recognition, it's 
a key point for machines to adapt themselves to complex and changing environment. In this paper, a new idea of concept 
acclimatization is presented, and we take the method of Multiple Stereovision and adaptive selection to solve the problem caused 
by posture change under the acclimation calculation framework. Also a decision level data fusion method, the combination of 
General Regression Neural Network and Dempster-Shafer Evidence Theory is presented to help improve accuracy of 
recognition. 

Keywords: Acclimatization; Multiple Stereovision; Data Fusion; General Regression Neural Network; Dempster-Shafer 
Evidence Theorv 



1. INTRODUCTION 

Face recognition is a research emphasis in image processing, 
pattern recognition, compared with other biometrics 
recognition technology, face recognition is more convenient, 
direct and easily accepted, it is widely used in authentication, 
video surveillance, criminal investigation and other fields. At 
present, most algorithms in face recognition are all for 
standard front face images. Even we ignore factors such as 
illumination, occlusion, the face images we get may have 
wide range of changes in the rotation angle, it is difficult to 
make a correct face recognition. So it is an important direction 
for us to have a accurate feature points location and 
recognition under various posture. 

Research shows that most methods for posture change are 
based on monocular vision system, such as multi-view 
Eigen-Face [1], efficient 3D reconstruction [2] .Although 
these methods can solve the problem in some degree, they 
need large amount of calculation and complex process. At the 
same time, changes of external environment is various, there 
are still many restrictions to solve these problems under 
monocular model. So we present the multi-vision method, get 
multiple view face images of one target with the use of 
multiple cameras. Also we can make use of redundant and 
complementary information from multi- sensors to improve 
the recognition rate. 

The remainder of the paper is organized as follows. In 
section 2, acclimation is defined, and the acclimation 
calculation framework under multi-vision is presented. 
Section 3 gives a detail introduction of the implementation 
method of acclimation calculation. Section 4 presents a 
decision level fusion algorithm which combines General 



Regression Neural Network with Dempster-Shafer Evidence 
Theory. Finally section 5 concludes the paper. 

2. ACCLIMATION CALCULATION UNDER 
MULTI-VISION 

2.1. Multiple Stereovision System and Acclimation 
Calculation 

Multiple Stereovision is to place multiple cameras to capture 
multi- view images of the same target, as shown in Figure 1, one 
of the cameras synchronizes with others. We can get more 
image information about target in Multi-Vision system than in 
monocular vision, which helps to reduce treatment difficulties, 
improves the accuracy and robustness of the detection system. 
Compared with binocular vision, increase in the number of 
cameras leads to an increase of geometric constraints, which 
reduces the difficulties of matching. 




Figure 1. Multiple Stereovision System 
In this paper, we define acclimation as the process that 
machines adapt themselves to external environment by means 
of detecting key environment parameters and adjusting 
system parameters automatically. The framework of 
acclimation calculation based on multiple stereovision mainly 
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contains three stages, input, pose estimation and acclimation 
calculation, as in Figure 2. In acclimation calculation, three 
steps are organized, key factor detection, calculation and 
decision. 
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Figure 2. Acclimation Calculation Framework 

2.2. Pose Estimation 

There are many methods for pose estimation, Choi [3] take 6 
feature points to estimate face rotation angle based on EM 
algorithm, it requires for long-time running. Lei Wang [4] take 
advantages of multiple cameras to locate 8 feature points and 
estimate rotation angles, this method needs camera calibration 
and matching, it's a large computing. Guoyuan Liang [5]use 
3D face model and affine correspondences to estimate face 
pose, it highly depends on the accuracy of 3D model data and 
it is complicated. For our method presented in the paper, we 
should first locate 5 feature points, the outer corner of left 
eye P Le , the outer corner of right eye P Re ,the nose tip P Np , the 

outer corner of left mouth P Lm , the outer corner of right 
mouthy, then estimate face pose ((p,y/,0) on the basis of 
projection theory. 

l)Face feature location 
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Figure 3.Build ASM model Figure 4. Feature points location 
We apply ASM(Active Shape Model)algorithm to locate face 
features. ASM( Active Shape Model)presented in [6] is a 
statistical model of the shape of objects which iteratively 
deform to fit to an example of the object in a new image, the 
shapes are constrained by the PDM(Point Distribution 
Model)to vary only in ways seen in a training set of labeled 
examples. The shape of an object is represented by a set of 



points. The ASM algorithm aims to match the model to a new 
image. It works by alternating the following steps. First, look 
in the image around each point for a better position for that 
point. Second, update the model parameters to best match to 
these new found positions. Figure 3 and Figure 4 show the 
process of ASM algorithm. Figure 5 gives a location image of 
68 feature points on the basis of ASM algorithm. 




Figure 5. Location of 68 feature points 
2)Rotation angle estimation 

• Expression of face posture 

Methods used to represent face pose are various, for example, 
use azimuth and dip angle, rotation angle in three-dimensional 
coordinate system and so on. In this paper, rotation angle are 
adopted. In rotation angle method, face is treated as 3D rigid 
body in three-dimensional coordinate system, images with 
various pose is regarded as 3D rigid body's projection in 
two-dimensional plane. Face is viewed as symmetric. Define 
the middle point of the connecting line of left and right outer 
eye corner O as origin of coordinates for both two-dimensional 
plane Oxy and three-dimensional coordinate system OXYZ ,see 
Figure 6. 



it 



Figure 6.Coordinate system in face image 
Without considering displacement of face image, the relation 
between a point p(x, y,z) in three-dimensional coordinate 
system and the corresponding one in two-dimensional plane 
P(X,Y) is computed as Eq.l,in which s is scale 
coefficients, and U is direct projective matrix Eq.2. 
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O is a transformation matrix. For front face image, O is unit 
matrix, while various face postures have different 
transformation matrix. According to rotation angle algorithm, 
various posture are equivalent to the front face's rotation 
about X , Y and Z axes, which can be presented by a set of 
different rotation angles (<p, y/, 6) ,so O can also be presented 
as 

<$> = R l {(p)R 2 {y/)R 3 {6) (3) 
In Eq.3, R { (<p) , R 2 (y/) , R 3 (0) respectively represents the 
rotation about X , Y , Z . 
• Estimation of posture angle 

In order to calculate the rotation angle (q?, y/,0) , we need know 

normal and staring direction of a face. We define the vertical 
line between nose tip and face as normal direction vector 

P nb P np ,the horizontal watch direction as staring direction 



vector P nb P ns .Take the method presented in [7] to estimate 
coordinates of 

points P ns (x 3 , y 3 , z 3 ) , P nb (x r , y x , z x ) , P np (x 2 , y 2 , z 2 ) ,as shown in 
Figure 7. 





Figure 7.Normal direction and staring direction of a face 
Rotation angles are computed as follows, 
a) Estimation of (p 

Suppose new coordinates of points P nb f (x/ , y/ , z[ ) and 

Pj (x 3 , y 3 ' , z 3 ) are got by rotating front face about X axis. The 
correlation can be computed as 
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Suppose new coordinates of points P nb " (x" , y" , z" ) and 

P n " ( x " ' y" ' z " ) are S ot by rotating front face about y axis. 
The correlation can be computed as 
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According to rotation relationship, y\ - y[ , y 3 = y 3 , after 



rotating about y axis, the staring direction P nb P m is parallel to 

z axis, x/' = x 3 " so Eq.7 is derived. 

x[ * cos y/-z{* sin y/ = x 3 * cos y/-z 3 ' * sin ^ (7) 
The rotation angle about y axis is obtained by solving Eq.7, 
coordinates of z" , z 3 are calculated at the same time, 
c) Estimation of 6 

After rotating about x, y axes, x/' = x 3 " , y/' = y 3 , the staring 
direction is parallel to z axis, finally, the rotation angle about 
z axis is obtained as Eq.8 

x" * cos 6 - y" * sin 6 = (8) 
2.3. Experiment Simulation 

Test our acclimatization calculation framework on ORL human 
face database. Human face pictures in ORL database are taken 
by Olivetti Libratory in University of Cambridge. The ORL 
human face database contains 40 volunteers' face images, in 
which each person is regarded as one class. Every class in ORL 
database has 10 gray images. These face images vary in the 
facial expression, lighting and posture. We divide 10 images of 
one person into two parts, one is training set, and the other is 
testing set. Testing set images are of the following two sorts, 
every sort has 3 test images and the remaining 4 images are set 
to be training images. 

1) Choose three images with smallest rotation angle ((p,y/,0) of 
the 10, as shown in Figure 8. 

2) Choose three images with random rotation angle of the 
remaining 7 images, as in Figure 9. 




Figure 8.Selected three images with minimum rotation angles 



In Eq.4, xi= x l , X3= x 3 .After rotating about x axis, the 

staring direction P nb P ns is parallel to Oxz plane, y/ = y 3 , so 
Eq.5 is derived. 

y 1 * cos cp - z x * sin <p = y 3 * cos cp - z 3 * sin cp (5) 
The rotation angle about x axis is obtained by solving Eq.5, 
coordinates of z^,z 3 are calculated on the basis of rotation 
relation. 

b) Estimation of y/ 
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Figure 9.Three images with random angle 
After training images and test images are assigned, Principal 
Component Analysis (PCA) and minimum distance decision 
are applied for face recognition at last. Recognition rate under 
these two situations are shown in Table 1. 

Table 1. Comparison of recogniton rate 



applied to assign basic probability and DS evidence theory is 
for decision, every basic probability assigned by GRNN is an 
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We can see from the results in Table 1 that when we take 
images with minimum rotation angles as test images, it has a 
great increse in recoginiton rate compared with images with 
random rotation angles. Results in Table 1 indicates that face 
images which have been selected by acclimatization calculation 
have more advantages on recognition rate than original images, 
it proves that the proposed acclimatization calculation 
framework is correct and practical to reduce the effect which is 
caused posture change. 

3. DATA FUSION BASED ON 

ACCLIMATIZATION CALCULATION 

As mentioned above, Multiple Stereovision system can obtain 
more information about targets than monocular vision. On the 
one hand ,we can select from multiple images to overcome the 
difficulty of posture change, on the other hand, images from 
multiple sensors can be used for data fusion. In Multiple 
Stereovision system ,all isomorphism or heterogeneous 
sensor's redundant and complementary information can be 
used for data fusion in certain level so as to make holistic 
assessment about targets more accurately. 

3.1. Multi-Sensor Information Fusion 

Multi- sensor data fusion technology can be divided into pixel 
level, feature level and decision level fusion [8]. Decision level 
fusion is to make final decision on the basis of certain principle 
and reliability of local decision, so there are two basic issues for 
decision fusion, one is decision principle, the other is how to 
characterize local reliability. Bayesian Probability Theory, 
Markovian Process,Dempster-Shafer Evidence Theory are all 
methods to estimate local reliability. Among them, 
Dempster-Shafer Evidence Theory is applied more widely due 
to its ability to describe uncertainty and combination rule. BP 
neural network is taken to combine with DS Evidence Theory 
in [9] for information fusion, while RBF is adopted in [10]. In 
this paper, we combine GRNN(General Regression Neural 
Network) with DS (Dempster-Shafer) evidence theory as fusion 
method in decision level, shown in Figure 10,where GRNN is 
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Figure lO.Fusion process based on f acclimatization calculation 

3.2. General Regression Neural Network 

GRNN(General Regression Neural Network) was proposed by 
Specht, it is a feedforward neural network which consists of 
input layer, hidden layer and output layer, see Figure 11. 




Figure 8.Structure of GRNN 
Figure ll.Structure of GRNN 

Compare with BP neural network, GRNN has more 
advantages 

1) Train for GRNN is one-way while BP neural network needs 
iteration 

2) Neurons numbers of hidden layer is determined by training 
samples adaptively 

3) Connection weight between layers uniquely determined by 
training samples while BP neural network modifies the weight 
in the process of iteration 

4) Gaussian function with local activation is adopted as 
activation function of hidden layer nodes, which has strong 
attraction to input close to local neurons features. 

3.3. Dempster-Shafer Evidence Theory 

Dempster-Shafer evidence theory was first put forward by 
Dempster in 1976 [10], it has the ability to describe uncertainty. 
In the application of DS evidence theory, the most important step is 
to estimate BPA (Basic Probability Assignment), definition of 
BPA is given as follows. 

A is an evidence for some proposition, Let be the 
framework of the proposition's identification, if function 
m : 2® e [0, 1] satisfies 

1) m(0) = 
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2)^>(A) = 1 



Then m(A) is the BPA (Basic Probability Assignment) of A, 
who stands for the support degree for evidence A. 
• Basic Probability Assignment for Face Recognition 
Suppose there are n objects faces F 1 ,F 2 ,...F n in our face library, 
so their identification framework = {F 19 F 2 ,...F n } , store their 
twelfth Zernike moments [11,12] in library, when selected 
images after acclimatization calculation of identifying target 
comes, extract their twelfth Zernike moments correspondingly. 
Suppose pattern x, y contain m eigenvalues 

X — \_X^ , %2 , — , X m ] 

y = [y lf y 2 ,....,y m ] 

Euclidean distance between x and y is 
d = ||x - y|| 

= ^(x l -y l ) 2 +(x 2 -y 2 ) 2 +...(x m -y m ) 2 (9) 

So the basic probability assignment of evidence i to target 
Fj is defined as 

m I .(F.) = (l-a)exp(-d..) (10) 

a is noise figure, a =0 when there is no noise in system, 
when identifying target is totally blanked by noise, 
a =1. Correspondingly, the BPA to uncertainty is 



m i (©) = fl(l-m i (F j )) 



(11) 



Before applying DS evidence theory, foregoing assigned 
BPA should be normalized as Eq.12. C is recognition rate of 
GRNN neural network. 



m i= c *yi /2>< i=1 > 2 - N 



(12) 



m(0) = l-C 
• Combination rule of DS Evidence Theory 
Suppose A and B are independent evidences in identification 

framework ,m 1 andm 2 are BPA to A, B, their orthogonal 
sum is described as 

/77(C) = /77, © m 2 
New BPA after combination is computed as Eq.13 
^m l (A)*m 2 (B) 



K 



(13) 



K = l- ^m 1 (A)*m 2 (5) 

Anfi=0 

= 2>(A)*m 2 (B) 

K calculated by Eq.14 is a factor who reflects the confliction 
degree between A and B. When K exceeds 1, DS evidence 
theory can not be applied. 
• Decision Rules 

Before data fusion, certain decision rules should be given, in 



this paper, we give the following rules. 

1) Target category should have the maximum BPA. 

2) Difference between target category and other category 
should be greater than some threshold A 2 . 

3) BPA to uncertainty must be less than some threshold X x that 
is the uncertainty for target category should not be too large. 

3.4. Experiment Simulation 

There are two parts in our simulation, one is selection part and 
the other is fusion part. The selection part has been proved in 
the second section of this paper. And in this section, we only 
take fusion part into account. 

Two images of each person are selected by acclimatization 
calculation in selection part for information fusion, whose 
decision rule is min 1 (<p + y/ + 6) , min 2 (cp + y/ + 6) .Then 

calculate the recognition result according to the information 
fusion method presented in this paper. 

Table 2, Table 3 and Table 4 illustrate the fusion process to 3 
classes of face in one cycle. M x , M 2 respectively stand for the 

selected two images of one person, GRNN 1 and GRNN 2 are 
general regression neural networks established for the selected 
two images, and F x , F 2 , F 3 stand for 3 classes of people face. 
Table 2. BPA assigned by GRNN 1 



Image 


Class of face images 


Name 











M x 


0.1763 


0.0869 


0.0535 


0.6833 


Table 3. BPA assigned by GRNN 2 


Image 


Class of face images 


Name 




F 2 







M 2 


0.3564 


0.2018 


0.1255 


0.3164 



Conflict factor between M, and M 2 is computed based on 
Eq.14 

K = M l (F l )*[M 2 (F 2 ) + M 2 (F 3 )] + M l (F 2 )* 

[M 2 (F, ) + M 2 (F 3 )] + M y (F 3 ) * [M 2 (F, ) + M 2 (F 2 )] 
= 0.1294 

After data fusion, BPA of M l , M 2 to 3 classes of face is 
computed as Eq.13 



M 12 (F 1 ) = 



M x (F x )*[M 1 (F 2 ) + M 1 {F,) + M 1 (F & )] + M x (F e )*M 2 (F x ) 
l-K 



M,(F 2 )*[M 2 (F I ) + M 2 (F 3 ) + M 2 (F 9 )] + M 1 (F 9 )*M 2 (F 2 ) 

a( 2 ' = jT^ 

M 1 (F 3 )*[M 2 (F 2 ) + M 2 (F 1 ) + M 2 (F e )] + M 1 (F e )*M 2 (F 3 ) 



M 12 (F 3 ) = 



l-K 



(14) M n {F @ ) = 



M 2 (F Q )*M,{F Q ) 



l-K 

Table 4. BPA assigned after information fusion 



Image 


Class of face images 


Name 


F x 


F 2 


^3 





M l2 


0.4101 


0.6481 


0.8003 


0.2162 
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From the data shown in Table 2, Table 3 and Table 4, we 

can conclude that the fusion method combing DS evidence 
theory and GRNN not only reduce the uncertainty in face 
recognition but also improve the accuracy of holistic judgments 
about target. 

Figure 12 and Figure 13 show the recognition result before 
and after data fusion . Figure 12 shows the recognition result 
of single image with minimum rotation angle in two selected 
images and Figure 13 shows the fusion recognition result of 
two selected images. In these two figures, the X-Coordinate 
stands for the real class of each test images while the 
Y-Coordinate stands for the predictive class of each image 
calculated by neural network. 
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Figure 12.Recognition of single image 

Fusion recognition of two images with minimum rotation angles 
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we will load the images selected by acclimatization calculation 
which are stored in the local file. Recognition methods are 
single image recognition and two images fusion recognition. 
We can choose one recognition method by pressing one of the 
radio buttons on the interface. At last, we display the matched 
recognition result. 
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Figure 14.1nterface of System 



True values of test images 

Figure 13.Fusion recognition of two images 
From the results, we can observe that there are 
misrecognitions in Figure 12, the face image of class 1 is 
recognized as class 2, and face image of class 5 is recognized as 
class 4. However, when we take the fusion result of two images 
for recognition, all classes are recognized rightly, as is shown in 
Figure 13. 

4. SYSTEM DESIGN 

Finally, we design a image fusion recognition system on the 
Matlab platform, as is shown in Figure 14.The whole system 
consists of three main parts, read images, choose recognition 
method and show the recognition result. For reading images, 
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Figure 16.Result of Two Images Fusion Recognition 

Figure 15 shows the result of single image recognition about 
one person while Figure 16 shows the recognition result of two 
images' fusion. It can be concluded that when taking two 
images and applying fusion method in face recognition, it can 

reduce the uncertainty and improve the accuracy 
of recognition, effectively 

5. CONCLUSION 

The experiment results in this paper have proved that applying 
Multiple Stereovision in face recognition system and taking 
acclimatization calculation method to select from input images 
can reduce the impact of posture change on recognition, in the 
meanwhile, redundant and complementary information from 
multi- sensors can be used for data fusion to improve the 
accuracy of identification. The simulation results prove our 
thinking to be right and practical. 
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